Search Results: "jonas"

17 July 2016

Vasudev Kamath: Switching from approx to apt-cacher-ng

After a long ~5 years (from 2011) journey with approx I finally wanted to switch to something new like apt-cacher-ng. And after a bit of changes I finally managed to get apt-cacher-ng into my work flow.
Bit of History I should first give you a brief on how I started using approx. It all started in MiniDebconf 2011 which I organized at my Alma-mater. I met Jonas Smedegaard here and from him I learned about approx. Jonas has a bunch of machines at his home and he was active user of approx and he showed it to me while explaining the Boxer project. I was quite impressed with approx. Back then I was using a 230kbps slow INTERNET connection and I was also maintaining a couple of packages in Debian. Updating the pbuilder chroots was time consuming task for me as I had to download multiple times over slow net. And approx largely solved this problem and I started using it. 5 years fast forward I now have quite fast INTERNET with good FUP. (About 50GB a month), but I still tend to use approx which makes building packages quite faster. I also use couple of containers on my laptop which all use my laptop as approx cache.
Why switch? So why change to apt-cacher-ng?. Approx is a simple tool, it runs mainly with inetd and sits between apt and the repository on INTERNET. Where as apt-cacher-ng provides a lot of features. Below are some listed from the apt-cacher-ng manual.
  • use of TLS/SSL repositories (may be possible with approx but I'm notsure how to do it)
  • Access control of who can access caching server
  • Integration with debdelta (I've not tried, approx also supports debdelta)
  • Avoiding use of apt-cacher-ng for some hosts
  • Avoiding caching of some file types
  • Partial mirroring for offline usage.
  • Selection of ipv4 or ipv6 for connections.
The biggest change I see is the speed difference between approx and apt-cacher-ng. I think this is mainly because apt-cacher-ng is threaded where as approx runs using inetd. I do not want all features of apt-cacher-ng at the moment, but who knows in future I might need some features and hence I decided to switch to apt-cacher-ng over approx.
Transition Transition from approx to apt-cacher-ng was smoother than I expected. There are 2 approaches you can use one is explicit routing another is transparent routing. I prefer transparent routing and I only had to change my /etc/apt/sources.list to use the actual repository URL. 
deb http://deb.debian.org/debian unstable main contrib non-free
deb-src http://deb.debian.org/debian unstable main
deb http://deb.debian.org/debian experimental main contrib non-free
deb-src http://deb.debian.org/debian experimental main
After above change I had to add a 01proxy configuration file to /etc/apt/apt.conf.d/ with following content. 
Acquire::http::Proxy "http://localhost:3142/"
I use explicit routing only when using apt-cacher-ng with pbuilder and debootstrap. Following snippet shows explicit routing through /etc/apt/sources.list. 
deb http://localhost:3142/deb.debian.org/debian unstable main
Usage with pbuilder and friends To use apt-cacher-ng with pbuilder you need to modify /etc/pbuilderrc to contain following line 
MIRRORSITE=http://localhost:3142/deb.debian.org/debian
Usage with debootstrap To use apt-cacher-ng with debootstrap, pass MIRROR argument of debootstrap as http://localhost:3142/deb.debian.org/debian.
Conclusion I've now completed full transition of my work flow to apt-cacher-ng and purged approx and its cache.
Though it works fine I feel that there will be 2 caches created when you use transparent and explicit proxy using localhost:3142 URL. I'm sure it is possible to configure this to avoid duplication, but I've not yet figured it. If you know how to fix this do let me know.
Update Jonas told me that its not 2 caches but 2 routing paths, one for transparent routing and another for explicit routing. So I guess there is nothing here to fix :-).

13 July 2016

Norbert Preining: Jonas Jonasson The Girl Who Saved the King of Sweden

Just finished my first book of Jonas Jonasson, a Swedish journalist and author. Most famous for his book The Hundred-Year-Old Man Who Climbed Out the Window and Disappeared, but author of two others. The one I read was The Girl Who Saved the King of Sweden, which strange enough became in German Die Analphabetin die rechnen konnte (The analphabet who could compute). Jonas Jonasson - Die Analphabetin, die rechnen konnte The story recounts the countless turns the life of Nombeko Mayeki, a black girl born in Soweto as latrine cleaner, who manages to save the Swedish king as well as most of the world from an atomic desaster by first getting driven over by a drunkard of South African nuclear bomb engineer, then meeting a clique of three Chinese sisters excelling in faking antiquities, and two Mossad agents. With the (unwilling) help of those agents she escapes to Sweden (including the atomic bomb) where she meets twins of a psychotic father who brought them up as one child so that the spare one can eradicate the Swedish monarchy. After many twists and setbacks, including several meetings with the Chinese premier Hu Jintao, she finally manages to get rid of the atomic bomb, get her undercover twin a real identity, and set up a proper life ah, and not to forget, save the King of Sweden! A fast paced, surprisingly funny and lovely story about how little things can change our lives completely.

3 July 2016

Reproducible builds folks: Reproducible builds: week 61 in Stretch cycle

What happened in the Reproducible Builds effort between June 19th and June 25th 2016. Media coverage GSoC and Outreachy updates Toolchain fixes Other upstream fixes Emil Velikov searched on IRC for hints on how to guarantee unique values during build to invalidate shader caches in Mesa, when also no VCS information is available. A possible solution is a timestamp, which is unique enough for local builds, but can still be reproducible by allowing it to be overwritten with SOURCE_DATE_EPOCH. Packages fixed The following 9 packages have become reproducible due to changes in their build dependencies: cclib librun-parts-perl llvm-toolchain-snapshot python-crypto python-openid r-bioc-shortread r-bioc-variantannotation ruby-hdfeos5 sqlparse The following packages have become reproducible after being fixed: Some uploads have fixed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Package reviews 139 reviews have been added, 20 have been updated and 21 have been removed in this week. New issues found: 53 FTBFS bugs have been reported by Chris Lamb, Santiago Vila and Mateusz ukasik. diffoscope development Quote of the week "My builds are so reproducible, they fail exactly every second time." Johannes Ziemke (@discordianfish) Misc. This week's edition was written by Chris Lamb (lamby), Reiner Herrmann and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

28 June 2016

Paul Wise: DebCamp16 day 5

Beat head against shiny cats (no animals were harmed). Discuss the spice of sillyness. Forward a wiki bounce to the person. Mention my gobby git mail cron job. Start adopting the adequate package. Discuss cats vs licensecheck with Jonas. Usual spam reporting. Review wiki RecentChanges. Whitelisted one user in the wiki anti-spam system. Finding myself longing for a web technology. Shudder and look at the twinklies.

17 May 2016

Reproducible builds folks: Reproducible builds: week 55 in Stretch cycle

What happened in the Reproducible Builds effort between May 8th and May 14th 2016: Documentation updates Toolchain fixes Packages fixed The following 28 packages have become newly reproducible due to changes in their build dependencies: actor-framework ask asterisk-prompt-fr-armelle asterisk-prompt-fr-proformatique coccinelle cwebx d-itg device-tree-compiler flann fortunes-es idlastro jabref konclude latexdiff libint minlog modplugtools mummer mwrap mxallowd mysql-mmm ocaml-atd ocamlviz postbooks pycorrfit pyscanfcs python-pcs weka The following 9 packages had older versions which were reproducible, and their latest versions are now reproducible again due to changes in their build dependencies: csync2 dune-common dune-localfunctions libcommons-jxpath-java libcommons-logging-java libstax-java libyanfs-java python-daemon yacas The following packages have become newly reproducible after being fixed: The following packages had older versions which were reproducible, and their latest versions are now reproducible again after being fixed: Some uploads have fixed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Package reviews 344 reviews have been added, 125 have been updated and 20 have been removed in this week. 14 FTBFS bugs have been reported by Chris Lamb. tests.reproducible-builds.org Misc. Dan Kegel sent a mail to report about his experiments with a reproducible dpkg PPA for Ubuntu. According to him sudo add-apt-repository ppa:dank/dpkg && sudo apt-get update && sudo apt-get install dpkg should be enough to get reproducible builds on Ubuntu 16.04. This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

1 February 2016

Lunar: Reproducible builds: week 40 in Stretch cycle

What happened in the reproducible builds effort between January 24th and January 30th:

Media coverage Holger Levsen was interviewed by the FOSDEM team to introduce his talk on Sunday 31st.

Toolchain fixes Jonas Smedegaard uploaded d-shlibs/0.63 which makes the order of dependencies generated by d-devlibdeps stable accross locales. Original patch by Reiner Herrmann.

Packages fixed The following 53 packages have become reproducible due to changes in their build dependencies: appstream-glib, aptitude, arbtt, btrfs-tools, cinnamon-settings-daemon, cppcheck, debian-security-support, easytag, gitit, gnash, gnome-control-center, gnome-keyring, gnome-shell, gnome-software, graphite2, gtk+2.0, gupnp, gvfs, gyp, hgview, htmlcxx, i3status, imms, irker, jmapviewer, katarakt, kmod, lastpass-cli, libaccounts-glib, libam7xxx, libldm, libopenobex, libsecret, linthesia, mate-session-manager, mpris-remote, network-manager, paprefs, php-opencloud, pisa, pyacidobasic, python-pymzml, python-pyscss, qtquick1-opensource-src, rdkit, ruby-rails-html-sanitizer, shellex, slony1-2, spacezero, spamprobe, sugar-toolkit-gtk3, tachyon, tgt. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them:
  • gnubg/1.05.000-4 by Russ Allbery.
  • grcompiler/4.2-6 by Hideki Yamane.
  • sdlgfx/2.0.25-5 fix by Felix Geyer, uploaded by Gianfranco Costamagna.
Patches submitted which have not made their way to the archive yet:
  • #812876 on glib2.0 by Lunar: ensure that functions are sorted using the C locale when giotypefuncs.c is generated.

diffoscope development diffoscope 48 was released on January 26th. It fixes several issues introduced by the retrieval of extra symbols from Debian debug packages. It also restores compatibility with older versions of binutils which does not support readelf --decompress.

strip-nondeterminism development strip-nondeterminism 0.015-1 was uploaded on January 27th. It fixes handling of signed JAR files which are now going to be ignored to keep the signatures intact.

Package reviews 54 reviews have been removed, 36 added and 17 updated in the previous week. 30 new FTBFS bugs have been submitted by Chris Lamb, Michael Tautschnig, Mattia Rizzolo, Tobias Frost.

Misc. Alexander Couzens and Bryan Newbold have been busy fixing more issues in OpenWrt. Version 1.6.3 of FreeBSD's package manager pkg(8) now supports SOURCE_DATE_EPOCH. Ross Karchner did a lightning talk about reproducible builds at his work place and shared the slides.

29 January 2016

Jan Wagner: Oxidized - silly attempt at (Really Awesome New Cisco confIg Differ)

Since ages I wanted have replaced this freaking backup solution of our Network Equipment based on some hacky shell scripts and expect uploading the configs on a TFTP server. Years ago I stumbled upon RANCID (Really Awesome New Cisco confIg Differ) but had no time to implement it. Now I returned to my idea to get rid of all our old crap.
I don't know where, I think it was at DENOG2, I saw RANCID coupled with a VCS, where the NOC was notified about configuration (and inventory) changes by mailing the configuration diff and the history was indeed in the VCS.
The good old RANCID seems not to support to write into a VCS out of the box. But for the rescue there is rancid-git, a fork that promises git extensions and support for colorized emails. So far so good. While I was searching for a VCS capable RANCID, somewhere under a stone I found Oxidized, a 'silly attempt at rancid'. Looking at it, it seems more sophisticated, so I thought this might be the right attempt. Unfortunately there is no Debian package available, but I found an ITP created by Jonas. Anyway, for just looking into it, I thought the Docker path for a testbed might be a good idea, as no Debian package ist available (yet). For oxidized configuration is only a configfile needed and as nodes source a rancid compatible router.db file can be used (beside SQLite and http backend). A migration into a production environment seems pretty easy. So I gave it a go. I assume Docker is installed already. There seems to be a Docker image on Docker Hub, that looks official, but it seems not maintained (actually). An issue is open for automated building the image.

Creating Oxidized container image The official documentation describes the procedure. I used a slightly different approach. 
docking-station:~# mkdir -p /srv/docker/oxidized/  
docking-station:~# git clone https://github.com/ytti/oxidized \  
 /srv/docker/oxidized/oxidized.git
docking-station:~# docker build -q -t oxidized/oxidized:latest \  
 /srv/docker/oxidized/oxidized.git
I thought it might be a good idea to also tag the image with the actual version of the gem. 
docking-station:~# docker tag oxidized/oxidized:latest \  
 oxidized/oxidized:0.11.0
docking-station:~# docker images   grep oxidized  
oxidized/oxidized   latest    35a325792078  15 seconds ago  496.1 MB  
oxidized/oxidized   0.11.0    35a325792078  15 seconds ago  496.1 MB
Create initial default configuration like described in the documentation. 
docking-station:~# mkir -p /srv/docker/oxidized/.config/  
docking-station:~# docker run -e CONFIG_RELOAD_INTERVAL=600 \  
 -v /srv/docker/oxidized/.config/:/root/.config/oxidized \
 -p 8888:8888/tcp -t oxidized/oxidized:latest oxidized

Adjusting configuration After this I adjusted the default configuration for writing a log, the nodes config into a bare git, having nodes secrets in router.db and some hooks for debugging.

Creating node configuration 
docking-station:~# echo "7204vxr.lab.cyconet.org:cisco:admin:password:enable" >> \  
 /srv/docker/oxidized/.config/router.db
docking-station:~# echo "ccr1036.lab.cyconet.org:routeros:admin:password" >> \  
 /srv/docker/oxidized/.config/router.db

Starting the oxidized beast 
docking-station:~# docker run -e CONFIG_RELOAD_INTERVAL=600 \  
 -v /srv/docker/oxidized/.config/:/root/.config/oxidized \
 -p 8888:8888/tcp -t oxidized/oxidized:latest oxidized
Puma 2.16.0 starting...  
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:8888
If you want to have the container get started with the docker daemon automatically, you can start the container with --restart always and docker will take care of it. If I wanted to make it running permanent, I would use a systemd unitfile.

Reload configuration immediately If you don't want to wait to automatically reload of the configuration, you can trigger it. 
docking-station:~# curl -s http://localhost:8888/reload?format=json \  
 -O /dev/null
docking-station:~# tail -2 /srv/docker/oxidized/.config/log/oxidized.log  
I, [2016-01-29T16:50:46.971904 #1]  INFO -- : Oxidized starting, running as pid 1  
I, [2016-01-29T16:50:47.073307 #1]  INFO -- : Loaded 2 nodes

Writing nodes configuration 
docking-station:/srv/docker/oxidized/.config/oxidized.git# git shortlog  
Oxidizied (2):  
      update 7204vxr.lab.cyconet.org
      update ccr1036.lab.cyconet.org
Writing the nodes configurations into a local bare git repository is neat but far from perfect. It would be cool to have all the stuff in a central VCS. So I'm pushing it every 5 minutes into one with a cron job. 
docking-station:~# cat /etc/cron.d/doxidized  
# m h dom mon dow user  command                                                 
*/5 * * * *    root    $(/srv/docker/oxidized/bin/oxidized_cron_git_push.sh)
docking-station:~# cat /srv/docker/oxidized/bin/oxidized_cron_git_push.sh  
#!/bin/bash
DOCKER_OXIDIZED_BASE="/srv/docker/oxidized/"  
OXIDIZED_GIT_DIR=".config/oxidized.git"
cd $ DOCKER_OXIDIZED_BASE /$ OXIDIZED_GIT_DIR   
git push origin master --quiet
Now having all the nodes configurations in a source code hosting system, we can browse the configurations, changes, history and even establish notifications for changes. Mission accomplished! Now I can test the coverage of our equipment. The last thing that would make me super happy, a oxidized Debian package!

27 January 2016

Jonas Smedegaard: BOSS - Barath Operating System Solutions

Siri and I are on a journey through India and Nepal, with the aim of learning about needs of Debian derivatives, to improve Debian and encourage closer integration.

C-DAC and BOSS Centre for Development of Advanced Computing (C-DAC) is a large organization serving country- and state-level institutions in India, with offices and training facilities several major cities. In Chennai, C-DAC has a staff of 25 developers working full time on Barath Operating System Solutions (BOSS). BOSS is a Debian derivative with several flavors - a desktop for use at primary schools (EduBOSS), a desktop for governmental offices (BOSS), and a range of server-oriented use cases using same core as the desktops with various (non-packaged) code and configuration on top. The core common to all BOSS flavors is a derivative of Debian. Major work has been in strengthening localization and related code - including the development of a font covering all officially supported indic scripts, tuning input methods configuration, and bugfixing LibreOffice handling of complex scripts. All that work is all passed directly to upstream code projects (some still show as derived work until sifting down again into Debian). Besides locale derivations, BOSS currently includes 11 packages not yet in Debian - a mixture of package dependencies, branding data and configuration tweaks. Seems most if not all can fit into Debian with a bit of restructuring work.

small computers As some of you know, I always had a special interest in low-resource (yet general purpose) computers (partly driven by my lack of money to spend on shinier hardware), and since ~2009 particularly in ARM-based computers. After 4 days of meetings and discussion with C-DAC, - literally few minutes before departure - I casually mentioned my interest in small computers, and much to my surprise it turned out that C-DAC also works on that, just didn't get around to mention it yet at the Debian wiki page. C-DAC have worked for a year on tuning BOSS to work on the Vidyut laptop (successor to the Aakash tablet). All except builtin camera is allegedly working. C-DAC is also looking into Olimex boards - my favorites - possibly for use with small server setups but our time was up, we had to leave for our train to Pune, so details on that we will have to figure out through mail.

collaboration In the past, C-DAC have kept in touch with their users through BOSS-specific places like a dedicated IRC channel. Recent changes in management style at the development office have caused less attention available to that communication, however. C-DAC have politely offered their code changes upstram for years, but maybe "too polite": Maybe they have offered only polished fixes, being less loud about "interesting problems". I suggested, as way to improve while limiting (ideally avoiding) extra work, is to mentally take a step up the stream: Treat BOSS not as a derivative but a subset of Debian itself, hang out and discuss issues and ideas at debian irc channels, and maintain your packages directly in Debian. Only parts unfit for Debian - secret stuff done for India military, and dirty configuration hacks not yet possible within Debian Policy - really need to be kept away from Debian. C-DAC agreed, and Debian now has a BOSS team! Anyone interested to follow BOSS as a Debian blend, and perhaps even contribute with opinions and/or code, is quite welcome to join the newly created mailinglist on Debian Alioth: https://lists.alioth.debian.org/mailman/listinfo/boss-devel. Our meetings with BOSS developers have been very pleasant. Even those working at the top of cloud or big data stacks - furthest away from our mindset of tightly "locking down" all parts as packages - were patient with us. Thanks in particular to Prema S and Prathibha B, working on packaging of BOSS for the past 5+ years, and both likely to enter the Debian New Maintainer Queue before long :-)

Jonas Smedegaard: India 2015

Siri and I are now three weeks into our two months journey, by train through India and by bus in Nepal. During my Asia 2011 journey I promised myself (and Chandan) that next visit to India would be together with Siri. Here we are, few hours away from next 20 hour train ride towards Hyderabad in South India, both with running noses from a cold week in Nepal. Theme of our trip is Debian Pure Blends. More specifically, we will meet with distribution developers and designers to try understand why they fork from (other forks of) Debian, and how Debian might improve to better serve them - ideally be able to fully contain such projects within Debian itself. Distributions we will look into - some more detailed than others - include Thanks to the organizations and individuals hosting us on our journey.

21 January 2016

Jonas Smedegaard: IT@school

Siri and I are on a journey through India and Nepal, with the aim of learning about needs of Debian derivatives, to improve Debian and encourage closer integration.

Distribution IT@school is a distribution originally based on Debian, later rebased on Ubuntu. Next release will possibly again be a direct derivative of Debian, or maybe even - time will tell - a Debian pure blend.

Aim is education As its name indicates, IT@school is targeted at schools: The system is used in 8th - 10th grades of most (or all?) public primary schools in Kerala, Together with KEK members Anto and Fayad, Siri and I met with former and current key participants in the project where we learned about its history and current status, and discussed some differences between Ubuntu and Debian. IT@school has a strong emphasis on the educational aspect, arguably setting it apart from Skolelinux/DebianEdu which emphasizes the technical aspect of relieving teachers from admin tasks. In the early years of deployment the project faced many hardware issues - e.g. in getting sound cards to work. This was seen not as problems but as beneficial learning for the teachers facing those issues. Kerala public school system has set the standard for other states in India, but sadly political support within the state has been weak in recent years. It is hoped that next election - this April - will bring a positive change.

School book IT@school is accompagnied by a school book written specifically for use of the included tools. No explicit license is applied to the book (which means it defaults to classic copyright). Possibly it will get Creative Commons licensed. If the school book gets a DFSG-free license, several collaboration opportunities emerge: Currently the book is drafted in LibreOffice but then - due to state procedures - finalized with PageMaker. Would be interesting to setup an alternate process using only Free tools - either with Scribus or XeLaTeX. An important detail here is to ensure that the process supports malayalam script.

Curriculum Work is in progress mapping FLOSS tools to the state curriculum. I recommended to share that work publicly with a Free license, to encourage comparisons across countries, and invite collaboration e.g. with Skolelunux/DebianEdu.

Blend for SBCs Some Kerala higher education schools (sorry, don't remember which) have bought some thousands of RaspberryPi2. I suggested to create a Debian Blend for SBCs (Single Board Computers) - we will see what comes of that idea

Blend for education I also suggested to make a Debian blend around IT@school distribution itself, with its strong focus on educational content - i.e. not just as addon to technical tools but the primary purpose pulling in tools as needed.

14 January 2016

Lunar: Reproducible builds: week 37 in Stretch cycle

What happened in the reproducible builds effort between January 3rd and January 9th 2016:

Toolchain fixes David Bremner uploaded dh-elpa/0.0.18 which adds a --fix-autoload-date option (on by default) to take autoload dates from changelog. Lunar updated and sent the patch adding the generation of .buildinfo to dpkg.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: aggressive-indent-mode, circe, company-mode, db4o, dh-elpa, editorconfig-emacs, expand-region-el, f-el, geiser, hyena, js2-mode, markdown-mode, mono-fuse, mysql-connector-net, openbve, regina-normal, sml-mode, vala-mode-el. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #809780 on flask-restful by Chris Lamb: implement support for SOURCE_DATE_EPOCH in the build system.
  • #810259 on avfs by Chris Lamb: implement support for SOURCE_DATE_EPOCH in the build system.
  • #810509 on apt by Mattia Rizzolo: ensure a stable file order is given to the linker.

reproducible.debian.net Add 2 more armhf build nodes provided by Vagrant Cascadian. This added 7 more armhf builder jobs. We now run around 900 tests of armhf packages each day. (h01ger) The footer of each page now indicates by which Jenkins jobs build it. (h01ger)

diffoscope development diffoscope 45 has been released on January 4th. It features huge memory improvements when comparing large files, several fixes of squashfs related issues that prevented comparing two Tails images, and improve the file list of tar and cpio archive to be more precise and consistent over time. It also fixes a typo that prevented the Mach-O to work (Rainer M ller), improves comparisons of ELF files when specified on the command line, and solves a few more encoding issues.

Package reviews 134 reviews have been removed, 30 added and 37 updated in the previous week. 20 new fail to build from source issues were reported by Chris Lamb and Chris West. prebuilder will now skip installing diffoscope to save time if the build results are identical. (Reiner Herrmann)

27 December 2015

Gregor Herrmann: RC bugs 2015/51-52

the upload of perl 5.22 to unstable some days ago provided ample opportunity to fix some new RC bugs. here's the list of my work:

Gregor Herrmann: RC bugs 2015/51-52

the upload of perl 5.22 to unstable some days ago provided ample opportunity to fix some new RC bugs. here's the list of my work:

22 December 2015

Jonas Smedegaard: Hamara Linux

Siri and I are on a journey through India and Nepal, with the aim of learning about needs of Debian derivatives, to improve Debian and encourage closer integration.

Distribution Hamara Linux is a distribution based on Trisquel, hence descending from Debian via Ubuntu. Next release will be a direct derivative of Debian. We recommended to package missing parts for Debian itself, even if Hamara needs them faster than deemed "stable" in Debian. ITP bugreports is since filed for theme and install routine.

Visual design Hamara Linux ships with a coherent visual style, covering widget theme, install routine, boot and login, and a range of wallpapers. Siri has begun comparing widget theme against Debian. We might try distill diffs for each Debian Ubuntu Trisquel Hamara derivation.

System contents and setup Hamara Linux comes in two flavors: Hamara Namaste with a GNOME desktop, and Hamara Sugam with an Lxde desktop. I have begun decomposing the package sets into classes for Boxer, at the same time extending Boxer.

21 December 2015

Jonas Smedegaard: SOME DESCRIPTIVE TITLE

# Copyright (C) YEAR Free Software Foundation, Inc. # This file is distributed under the same license as the PACKAGE package. # FIRST AUTHOR , YEAR. # #, fuzzy msgid "" msgstr "" "Project-Id-Version: PACKAGE VERSION\n" "POT-Creation-Date: 2015-12-21 15:00+0100\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME \n" "Language-Team: LANGUAGE \n" "Language: \n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" #. type: Title # #, no-wrap msgid "I want your input" msgstr "" #. type: Plain text #, no-wrap msgid "\n" msgstr "" #. type: Plain text #, no-wrap msgid "\n" msgstr "" #. type: Plain text msgid "You want to share an opinion with me, or raise a question?" msgstr "" #. type: Plain text msgid "Great, please do: I love exchanging opinions and ideas." msgstr "" #. type: Plain text msgid "" "You can [contact me discretely][]. Please consider, however, to speak in " "public instead, allowing others to benefit from our exchange." msgstr "" #. type: Plain text msgid "" "Don't worry if it is interesting to others - the World is big and someone " "somewhere always cares!" msgstr "" #. type: Plain text msgid "[contact me discretely]:" msgstr "" #. type: Title ## #, no-wrap msgid "Email - encourages reflection" msgstr "" #. type: Plain text msgid "Post to a public mailinglist." msgstr "" #. type: Plain text msgid "" "If uncertain that I am subscribed to the list, or if you want my special " "attention (but think twice if that's really needed!), add my [private " "mail](mailto:dr@jones.dk) as cc." msgstr "" #. type: Title ## #, no-wrap msgid "Chat - more casual" msgstr "" #. type: Plain text msgid "" "I am also [reachable via chat][]. Chat is nice for shorter " "questions/comments, and when you need to discuss what you want to say at " "all." msgstr "" #. type: Plain text msgid "[reachable via chat]:" msgstr "" #. type: Title ## #, no-wrap msgid "Blog - best for reuse" msgstr ""

Jonas Smedegaard: SOME DESCRIPTIVE TITLE

# Copyright (C) YEAR Free Software Foundation, Inc. # This file is distributed under the same license as the PACKAGE package. # FIRST AUTHOR , YEAR. # #, fuzzy msgid "" msgstr "" "Project-Id-Version: PACKAGE VERSION\n" "POT-Creation-Date: 2015-12-21 15:00+0100\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME \n" "Language-Team: LANGUAGE \n" "Language: \n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" #. type: Title # #, no-wrap msgid "issues handling issues" msgstr "" #. type: Plain text #, no-wrap msgid "\n" msgstr "" #. type: Plain text #, no-wrap msgid "\n" msgstr "" #. type: Plain text msgid "" "I get lots of emails, some of which are "issues": Friends, colleagues, " "clients and machines tell me about stuff that is broken. and ask for new " "features." msgstr "" #. type: Plain text msgid "I currently handle issues somewhat like this:" msgstr "" #. type: Plain text #, no-wrap msgid "[[!graph src="""\n" msgstr "" #. type: Plain text #, no-wrap msgid "" "rankdir=LR\n" "inbox [label="incoming mails"]\n" "todobox [label="mail threads about issues"]\n" "issueboxes [label="issue-mails grouped by project" style=dotted]\n" "solve [label="fix issue"]\n" "archivebox [label="archived mails"]\n" "inbox -> todobox\n" "todobox -> issueboxes [style=dotted]\n" "todobox -> solve\n" "todobox -> archivebox\n" msgstr "" #. type: Plain text msgid "FIXME: Describe graf in steps." msgstr "" #. type: Plain text msgid "" "The idea was to organize issues in queues per project, but organizing emails " "requires understanding them, and that takes time: Too often I end up " "skipping that task, leading to weak overview of pending issues." msgstr "" #. type: Plain text msgid "Here's" msgstr "" #. type: Plain text #, no-wrap msgid "" "rankdir=LR\n" "inbox [label="incoming mails"]\n" "todobox [label="mail threads about issues"]\n" "new [label="unprocessed issues"]\n" "open [label="open issues"]\n" "solve [label="fix issue"]\n" "inbox -> todobox -> archivebox\n" "todobox -> new -> open\n" "open -> solve\n" "open -> done\n" msgstr ""

13 December 2015

Vasudev Kamath: Mount ordering and automount options in systemd

Just to give a brief recap of why I'm writing this post, I've to describe a brief encounter I had with systemd which rendered my system unbootable. I first felt its systemd's problem but later figured out it was my mistake. So below is brief story divided into 2 sections Problem and Solution. Here Problem describes issue I faced with systemd and Solution discusses the .mount and .automount suffix files used by systemd.
Problem I have several local bin folders and I don't want to modify PATH environment variable adding every other folder path. I first started using aufs as alternative to symlinks or modifying PATH variable. Recently I learnt about much more light weight union fs which is in kernel, overlayfs. And Google led me to Arch wiki article which showed me a fstab entry like below 
overlay        /usr/local/bin        overlay noauto,x-systemd.automount,lowerdir=/home/vasudev/Documents/utilities/bin,upperdir=/home/vasudev/Documents/utilities/jonas-bin,workdir=/home/vasudev/Documents/utilities/bin-overlay    0       0
And after adding this entry, on next reboot systemd is not able to mount my LVM home and swap partition. It does however mount root partition. It gives me emergency target but login never returns. So without any alternative I had to reinstall the system. But funnily enough I started encountering the same issue (yeah after I had added above entry into fstab). Yeah that time I never thought that its culprit. My friend Ritesh finally got my system booting after the weird x-systemd.automount option. I never investigated further that time on why this problem occurred.
Solution After re-encounter with similar problem and some other project I read the manual on systemd.mount, systemd-fstab-generator and systemd.automount and I had some understanding of what really went wrong in my case above. So now let us see what really is happening. All the above happened because, systemd translates the /etc/fstab into .mount units at run time using systemd-fstab-generator. Every entry in fstab translates into a file named after the mount point. The / in the path of the mount point is replaced by a -. So / mount point is named as -.mount and /home is named as home.mount and /boot becomes boot.mount. All these files can be seen in directory /run/systemd/generator. And all these mount points are needed by local-fs.target , if any of these mount points fail, local-fs.target fails. And if local-fs.target fails it will invoke emergency.target. systemd-fstab-generator manual suggests the ordering information in /etc/fstab is discarded. That means if you have union mounts, bind mounts or fuse mounts in the fstab which is normally at end of fstab and uses path from /home or some other device mount points it will fail to get mounted. This happens because systemd-fstab-generator didn't consider the ordering in fstab. This is what happened in my case my overlay mount point which is usr-local-bin.mount was some how happened to be tried before home.mount because there is no explicit dependency like Requires= or After= was declared. When systemd tried to mount it the path required under /home were not really present hence failed which in return invoked emergency.target as usr-local-bin.mount is Requires= dependency for local-fs.target. Now what I didn't understand is why emregency.target never gave me the root shell after entering login information. I feel that this part is unrelated to the problem I described above and is some other bug. To over come this what we can do is provided systemd-fstab-generator some information on dependency of each mount point. systemd.mount manual page suggests several such options. One which I used in my case is x-systemd.requires which should be placed in options column of fstab and specify the mount point which is needed before it has to be mounted. So my overlay fs entry translates to something like below 
overlay        /usr/local/bin        overlay noauto,x-systemd.requires=/home,x-systemd.automount,lowerdir=/home/vasudev/Documents/utilities/bin,upperdir=/home/vasudev/Documents/utilities/jonas-bin,workdir=/home/vasudev/Documents/utilities/bin-overlay   0       0
There is another special option called x-systemd.automount this will make systemd-fstab-generator to create a .automount file for this mount point. What does systemd.automount do? It achieves on-demand file mounting and parallelized file system mounting. Something similar to systemd.socket the file system will be mounted when you access the mount point for first time. So now if you try to see dependency of usr-local-bin.mount you will see following. 
systemctl show -p After usr-local-bin.mount
After=-.mount systemd-journald.socket local-fs-pre.target system.slice usr-local-bin.automount
This means now usr-local-bin.mount depends on usr-local-bin.automount. And let us see what usr-local-bin.automount needs. 
systemctl show -p Requires usr-local-bin.automount
Requires=-.mount home.mount
systemctl show -p After usr-local-bin.automount
After=-.mount home.mount
So clearly usr-local-bin.automount is activated after -.mount and home.mount is activated. Similar can be done for any bind mounts or fuse mounts which require other mount points before it can be mounted. Also note that x-systemd.autmount is not mandatory option for declaring dependencies, I used it just to make sure /usr/local/bin is mounted only when its really needed.
Conclusion A lot of traditional way has been changed by systemd. I never understood why my system failed to boot in first place this happened because I was not really aware of how systemd works and was trying to debug the problem with traditional approach. So there is really a learning curve involved for every sysadmin out there who is going to use systemd. Most of them will read documentation before hand but others like me will learn after having a situation like above. :-). So systemd interfering into /etc/fstab is good?. I don't know but since systemd parallelizes the boot procedure something like this is really needed. Is there a way to make systemd not touch /etc/fstab?. Yes there is you need to pass fstab=0 option in kernel command line and systemd-fstab-generator doesn't create any .mount or .swap files from your /etc/fstab. !NB Also it looks like x-systemd.requires option was introduced recently and is not available in systemd <= 215 which is default in Jessie. So how do you declare dependencies in Jessie system?. I don't have any answer!. I did read that x-systemd.automount which is available in those versions of systemd can be used, but I'm yet to experiment this. If it succeeds I will write a post on it.

10 August 2015

Lunar: Reproducible builds: week 15 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Guillem Jover uploaded dpkg/1.18.2 which makes dependency comparisons deep by comparing not only the first dependency alternative, to get them sorted in a reproducible way. Original patch by Chris Lamb. Dhole updated the patch adding support for SOURCE_DATE_EPOCH in gettext. A modified package is in the experimental reproducible repository. Valentin Lorentz submitted a patch adding support for SOURCE_DATE_EPOCH to ocamldoc. Valentin Lorentz also opened a bug about the inability to set an arbitrary RNG seed for ocamlopt which would be a way to fix an issue affecting many OCaml packages. Dhole submitted a patch adding support for SOURCE_DATE_EPOCH in qhelpgenerator. A modified package has been sent to the experimental repository as well. Several packages have been updated for the experimental toolchain: doxygen (akira), and dpkg (h01ger). Also, h01ger has built and uploaded all experimental packages having arch:any packages for armhf: dpkg, gettext, doxygen, fontforge, libxslt and texlive-bin. We are now providing our toolchain for armhf and amd64. Packages fixed As you might have noticed, Debian sid is currently largely uninstallable, due to the GCC 5 transition, which also can be see in our reproducibility test setup. Please help! The following packages became reproducible due to changes in their build dependencies: glosstex, indent, ktikz, liblouis, libmicrohttpd, linkchecker, multiboot, qterm, rrep, trueprint, twittering-mode. The following packages became reproducible after getting fixed: Patches submitted which have not made their way to the archive yet: Lunar reported an issue on an unstable ABI from a generated header in icedove reminding of an issue affecting libical-dev. The bug has since been fixed by Carsten Schoenert. akira identified an unreferenced embeded code copy (causing unreproducibility!) in gperf. reproducible.debian.net The scheduler has temporarily been changed to not schedule any already tested packages for sid and experimental, due to the the GCC 5 transitions, which are well visible in our graphs now. On the plus side this has caused our stretch testing to catch up (and improve stats). (h01ger) depwait packages (packages where the Build-Depends cannot be satisfied) are now listed in the last 24h and last 48h pages (Mattia Rizzolo) Two new amd64 build nodes (with 8 cores and 32 GB RAM each) have been added, kindly sponsored by Profitbricks. (h01ger) The 4 armhf (setup last week by Vagrant Cascadian) and 2 amd64 build nodes have been made available to Jenkins. Remote job scheduling has been implemented and 35 new jobs have been added for pbuilder and schroot creation and maintenance of the nodes. (h01ger) The manual scheduler gained a flag (-a/--architecture) to select which arch to schedule in. (Mattia Rizzolo) armhf will only be testing stretch for now, due to limited hardware ressources. (h01ger) The page listing maintainers of unreproducible packages gained internal anchors. As an example, one can now link to unreproducible orphaned packages. (Mattia Rizzolo) Packages with a bug tagged pending are marked using a new symbol: a brown P (Mattia Rizzolo) diffoscope development debbindiff is now called diffoscope! It also has a website at diffoscope.org. The name was changed to better reflect that it became a general purpose tool, capable of comparing many different archive formats, or directories. Version 29 is the renaming release. Amongst a couple of other cosmetic changes a favicon showing the new logo has been added to the generated HTML reports. Version 30 replaces the file matching algorithm for files listed in .changes to a smarter one that removes only the version number. It also fixes a bug where squashfs directories were being extracted even if their content was being compared at a later stage. It also fixes an issue with the test suite that was detected by debci. Documentation update More rationale have been added for supporting SOURCE_DATE_EPOCH The unfinished Reproducible Builds HOWTO is now visible on the web, feedback and patches most welcome. Package reviews 261 obsolete reviews have been removed, 73 added and 145 updated this week.

3 August 2015

Lunar: Reproducible builds: week 14 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes akira submitted a patch to make cdbs export SOURCE_DATE_EPOCH. She uploded a package with the enhancement to the experimental reproducible repository. Packages fixed The following 15 packages became reproducible due to changes in their build dependencies: dracut, editorconfig-core, elasticsearch, fish, libftdi1, liblouisxml, mk-configure, nanoc, octave-bim, octave-data-smoothing, octave-financial, octave-ga, octave-missing-functions, octave-secs1d, octave-splines, valgrind. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: In contrib, Dmitry Smirnov improved libdvd-pkg with 1.3.99-1-1. Patches submitted which have not made their way to the archive yet: reproducible.debian.net Four armhf build hosts were provided by Vagrant Cascadian and have been configured to be used by jenkins.debian.net. Work on including armhf builds in the reproducible.debian.net webpages has begun. So far the repository comparison page just shows us which armhf binary packages are currently missing in our repo. (h01ger) The scheduler has been changed to re-schedule more packages from stretch than sid, as the gcc5 transition has started This mostly affects build log age. (h01ger) A new depwait status has been introduced for packages which can't be built because of missing build dependencies. (Mattia Rizzolo) debbindiff development Finally, on August 31st, Lunar released debbindiff 27 containing a complete overhaul of the code for the comparison stage. The new architecture is more versatile and extensible while minimizing code duplication. libarchive is now used to handle cpio archives and iso9660 images through the newly packaged python-libarchive-c. This should also help support a couple other archive formats in the future. Symlinks and devices are now properly compared. Text files are compared as Unicode after being decoded, and encoding differences are reported. Support for Sqlite3 and Mono/.NET executables has been added. Thanks to Valentin Lorentz, the test suite should now run on more systems. A small defiency in unquashfs has been identified in the process. A long standing optimization is now performed on Debian package: based on the content of the md5sums control file, we skip comparing files with matching hashes. This makes debbindiff usable on packages with many files. Fuzzy-matching is now performed for files in the same container (like a tarball) to handle renames. Also, for Debian .changes, listed files are now compared without looking the embedded version number. This makes debbindiff a lot more useful when comparing different versions of the same package. Based on the rearchitecturing work has been done to allow parallel processing. The branch now seems to work most of the time. More test needs to be done before it can be merged. The current fuzzy-matching algorithm, ssdeep, has showed disappointing results. One important use case is being able to properly compare debug symbols. Their path is made using the Build ID. As this identifier is made with a checksum of the binary content, finding things like CPP macros is much easier when a diff of the debug symbols is available. Good news is that TLSH, another fuzzy-matching algorithm, has been tested with much better results. A package is waiting in NEW and the code is ready for it to become available. A follow-up release 28 was made on August 2nd fixing content label used for gzip2, bzip2 and xz files and an error on text files only differing in their encoding. It also contains a small code improvement on how comments on Difference object are handled. This is the last release name debbindiff. A new name has been chosen to better reflect that it is not a Debian specific tool. Stay tuned! Documentation update Valentin Lorentz updated the patch submission template to suggest to write the kind of issue in the bug subject. Small progress have been made on the Reproducible Builds HOWTO while preparing the related CCCamp15 talk. Package reviews 235 obsolete reviews have been removed, 47 added and 113 updated this week. 42 reports for packages failing to build from source have been made by Chris West (Faux). New issue added this week: haskell_devscripts_locale_substvars. Misc. Valentin Lorentz wrote a script to report packages tested as unreproducible installed on a system. We encourage everyone to run it on their systems and give feedback!

28 February 2015

Mathieu Parent: Hello Planet Debian

After more than five years of being a Debian developer, here is my first post on Planet Debian! I currently maintain 165 packages. My focus has changed since 2009, but those are still mostly sysadmin packages: Now that the first post is online, I will try to keep up!

Next.

Previous.